Papers with manual labeling
HypoTermQA: Hypothetical Terms Dataset for Benchmarking Hallucination Tendency of LLMs (2024.eacl-srw)
Copied to clipboard
| Challenge: | Hallucinations pose a significant challenge to the reliability and alignment of Large Language Models (LLMs), limiting their widespread acceptance beyond chatbot applications. |
| Approach: | They propose a framework that combines benchmarking LLMs’ hallucination tendencies with efficient hallucinian detection. |
| Outcome: | The proposed framework provides opportunities to test and improve LLMs and can generate benchmarking datasets tailored to specific domains. |
Assessing In-context Learning and Fine-tuning for Topic Classification of German Web Data (2024.acl-srw)
Copied to clipboard
| Challenge: | Using a few hundred annotated data points per topic, we detect content related to three German policies in a database of scraped webpages. |
| Approach: | They propose to use annotated data to train a binary classification task to detect topic-related content in a scraped database of webpages. |
| Outcome: | The proposed model detects content related to three German policies in a scraped database of scrapes of webpages using a few hundred annotated data points per topic. |
CoLeM: A framework for semantic interpretation of Russian-language tables based on contrastive learning (2025.acl-srw)
Copied to clipboard
| Challenge: | Existing methods for semantic interpretation of tables in Russian lack explicit semantics . tabular data are one of the key formats for presenting structured information in various domains ranging from scientific research to business analytics. |
| Approach: | They propose a contrastive learning approach to minimize reliance on manual labeling and enhance the accuracy of column annotation for rare semantic types. |
| Outcome: | The proposed method minimizes reliance on manual labeling and improves column annotation accuracy for rare semantic types. |
AutoPRM: Automating Procedural Supervision for Multi-Step Reasoning via Controllable Question Decomposition (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have shown promise in multi-step reasoning tasks, yet relying on extensive manual labeling to provide procedural feedback remains a significant impediment. |
| Approach: | They propose a self-supervised framework that decomposes complex problems into manageable subquestions with a controllable granularity switch and sequentially applies reinforcement learning to iteratively improve the subquest solver. |
| Outcome: | The proposed framework improves performance on mathematical and commonsense reasoning tasks over SOTA. |
Generalizing Few-Shot Named Entity Recognizers to Unseen Domains with Type-Related Features (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Few-shot named entity recognition methods struggle with out-of-domain (OOD) examples due to their reliance on manual labeling for the target domain. |
| Approach: | They propose a framework to enable generalization to an unseen target domain with only a few labeled examples. |
| Outcome: | The proposed framework achieves significant performance improvements on in-domain and cross-domain datasets. |
Automatic Construction of a Chinese Review Dataset for Aspect Sentiment Triplet Extraction via Iterative Weak Supervision (2024.lrec-main)
Copied to clipboard
| Challenge: | Aspect Sentiment Triplet Extraction (ASTE) is a task that involves the extraction of three key elements: target aspects, descriptive opinion spans, and their corresponding sentiment polarity. |
| Approach: | They propose a framework that facilitates automatic construction of Aspect Sentiment Triplet Extraction (ASTE) by iterative weak supervision and a discriminator to weed out subpar samples. |
| Outcome: | The proposed framework automates the construction of Aspect Sentiment Triplet Extraction tasks in Chinese by using iterative weak supervision. |
Multi-label and Multi-target Sampling of Machine Annotation for Computational Stance Detection (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for data-driven annotations require domain-specific and task-aligned supervision. |
| Approach: | They propose a multi-label and multi-target sampling strategy to optimize the annotation quality. |
| Outcome: | The proposed method significantly improves performance and learning efficacy on the benchmark stance detection corpora. |
FLiText: A Faster and Lighter Semi-Supervised Text Classification with Convolution Networks (2021.emnlp-main)
Copied to clipboard
| Challenge: | obtaining large amounts of labeled data is expensive. |
| Approach: | They develop a semi-supervised learning framework called FLiText which improves text classification accuracy. |
| Outcome: | The proposed framework improves accuracy of lightweight models on IMDb, Yelp-5, and Yahoo! Answer . the framework improve accuracy by 6.59%, 3.94%, and 3.22% on the datasets of IMDa, Yep-5 and Yahoo. Answer compared with the fully supervised method on the full dataset . |
Topic-driven Ensemble for Online Advertising Generation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for automating online advertising use open data . subdomains of text vary in use and can lead to reduced quality of adverts generation. |
| Approach: | They propose a neural network-based approach for the automatic generation of online advertising using texts from given webpages as sources. |
| Outcome: | The proposed approach significantly improves the quality of online advertising generated on a Russian dataset. |
ACT-SQL: In-Context Learning for Text-to-SQL with Automatically-Generated Chain-of-Thought (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have focused on the development of semantic parsers within the framework of cross-domain analysis. |
| Approach: | They propose a method to generate auto-CoT exemplars using ACT-SQL and extend it to multi-turn text-to-Sql tasks. |
| Outcome: | The proposed method achieves SOTA performance on the Spider dev set among existing in-context learning approaches. |
ULF: Unsupervised Labeling Function Correction using Cross-Validation for Weak Supervision (2023.emnlp-main)
Copied to clipboard
| Challenge: | A cost-effective alternative to manual data labeling is weak supervision (WS), where data samples are automatically annotated using a predefined set of labeling functions (LFs). |
| Approach: | They propose an algorithm which denoises WS data by leveraging models trained on all but some LFs to identify and correct biases specific to the held-out LF. |
| Outcome: | The proposed algorithm denoises WS data by leveraging models trained on all but some LFs to identify and correct biases specific to the held-out LF. |
Named Entity Recognition through Deep Representation Learning and Weak Supervision (2021.findings-acl)
Copied to clipboard
| Challenge: | Weakly supervised named entity recognition (NER) uses noisy labels to estimate the true labels of a dataset. |
| Approach: | They propose a model to learn optimal assignments of latent NER tags using observed tokens and weak labels provided by labeling functions. |
| Outcome: | The proposed model improves the quality of weak labels on four public datasets. |
Automated Molecular Concept Generation and Labeling with Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Concept-based models lack explainability and need predefined concepts and manual labeling in molecular science. |
| Approach: | They propose a framework that leverages Large Language Models to generate and label predictive molecular concepts without human input. |
| Outcome: | The proposed framework outperforms existing models on several benchmarks while maintaining explainability and allowing easy intervention. |
Empirical Analysis of Task Mixture Effects in Small-scale Instruction Tuning: A Statistical Approach (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that instruction tuning can significantly vary depending on the task and format diversity of training data. |
| Approach: | They conduct experiments on unlabeled instruction corpora to identify effective mixtures. |
| Outcome: | The proposed model can achieve up to 5.7 speedup in training with 1,000 curated examples. |
TOXIFRENCH: Benchmarking and Enhancing Language Models via CoT Fine-Tuning for French Toxicity Detection (2026.findings-acl)
Copied to clipboard
| Challenge: | toxicity detection in French remains underdeveloped due to the lack of culturally relevant, human-annotated, large-scale datasets. |
| Approach: | They propose a method that generalizes French online comments using a semi-automated annotation pipeline that reduces manual labeling to only 10% through high-confidence LLM-based pre-annotation and human verification. |
| Outcome: | The proposed model outperforms GPT-4o and DeepSeek-R1 on the benchmark while maintaining cross-lingual capabilities. |
Prototype-based Prompt-Instance Interaction with Causal Intervention for Few-shot Event Detection (2024.lrec-main)
Copied to clipboard
| Challenge: | Few-shot Event Detection (FSED) requires limited labeled data and expensive manual labeling. |
| Approach: | They propose a prototype-based prompt-instance Interaction with causal Intervention model to utilize both prompts and verbalizers and effectively eliminate all biases. |
| Outcome: | The proposed model utilizes both prompts and verbalizers and eliminates all biases on RAMS and ACE datasets. |
Inference Helps PLMs’ Conceptual Understanding: Improving the Abstract Inference Ability with Hierarchical Conceptual Entailment Graphs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to abstract inference ignore the *polysemy* and *hierarchical nature of concepts* . prevailing approaches disregard how arguments might entail differently across various concept levels, thereby missing potential enlargement connections. |
| Approach: | They propose a framework that organizes arguments hierarchically and delves into entailment relations at diverse concept levels. |
| Outcome: | The proposed framework improves the model's generalization and reasoning prowess in natural language inference. |
HICode: Hierarchical Inductive Coding with LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for fine-grained corpus analysis are limited to small subsets and require manual labeling and statistical tools like topic modeling. |
| Approach: | They propose a pipeline that inductively generates labels from analysis data and then hierarchically clusters them to surface emergent themes. |
| Outcome: | The proposed pipeline validates the approach across three datasets and shows it is robust through automated and human evaluations. |